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© Image processing apparatus. 

© The invention is intended to improve image quality in a process of image coding. In an apparatus for 
quantizing conversion data resulted from converting image information into frequency regions and then coding 
the quantized data, a coefficient to be multiplied by a chrominance quantizing matrix is set depending on a 
coefficient to be multiplied by a luminance quantizing matrix for an improvement in color reproducibility. 



FIG.1 



101 

/ . 



104 



^102 


^103 f 


NxN 




UN EAR 




dctckt 




QUANTIZER 





| PREDICTOR 



^,05 


HUFFMAN 




ENCODER 


f 



.108 



.106 



.107 



115 



X. 



ZIGZAG 




HUFFMAN 




SCAN CKT 




ENCODER 





MULTIPLEXER -6 



109 



SCALING FACTOR 
GENERATION CKT 



,113 



-114 



! a b 



LUMINANCE CHROMINANCE 



QUANTIZATION MATRIX 
GENERATION CKT 



CONTROL 
CKT 



/12 116 
' 6 



Rank Xerox (UK) Business Services 

•3.10/3.6/3.3." 



J 

EP 0 566 219 A2 



The present invention relates to an image processing method and apparatus, and more particularly to 
an image processing method and apparatus for quantizing frequency conversion data and coding the 
quantized conversion data. 

A number of color image compressing techniques have been proposed in the past. As a typical one of 

5 color still image coding techniques, there is proposed an adaptive discrete cosine transform coding method 
(hereinafter referred to as an ADCT method). 

Fig. 3 is a conceptual view showing arrangement of an image coding apparatus using the ADCT 
method. It is assumed that an input image is given by data converted into 8 bits, i.e., 256 gradations, for 
each color in a not-shown analog-to-digital converter (hereinafter referred to as an A/D converter) and the 

w number of colors is three, i.e.. luminance Y, color P 8 and color P R . in this case. 

The digital data of the luminance signal Y inputted through an input terminal 101 in Fig. 3 is subjected 
to two-dimensional discrete cosine transform (hereinafter abbreviated as OCT) in an N x N DCT circuit 102 
in units of subblocks each comprising N x N pixels. Thereafter, the conversion coefficients are linearly 
quantized in a linear quantizer 113 with a quantizing step size being different for each of the conversion 

is coefficients. The quantizing step sizes for respective conversion coefficients are given by values resulted 
from multiplying 2 s by quantization matrix elements of N x N in consideration of the fact that visual 
sensitivity for quantizing noise is different for each of the conversion coefficients. Here. S is 0 or a positive 
or negative integer and called a scaling factor. The image quality or the data to be generated is controlled 
depending on the value of S. 

20 One example of the quantization matrix elements is shown in Fig. 5 in the case of 8 x 8. A quantization 
matrix generation circuit 111 is controlled by a control circuit 302 and generates a quantization matrix for 
the luminance signal Y. A switch 114 is changed over to select the side a for supplying the quantization 
matrix for the luminance signal Y to a multiplier 110. Using the scaling factor S supplied from a scaling 
factor generation circuit 301. the multiplier 110 multiplies 2 s by each quantized element of the quantization 

25 matrix. The multiplied results are supplied to the linear quantizer 103 where the conversion coefficients are 
linearly quantized by using the supplied results. 

After the quantization, direct current conversion coefficients (hereinafter referred to as DC components) 
are supplied to a one-dimensional predictor 104 to calculate prediction values using several subblocks 
adjacent to each other, and prediction errors are subjected to Huffman coding in a Huffman encoder 105. 

30 More specifically, after dividing quantized outputs of the prediction errors into groups, the ID numbers of the 
groups to which the respective prediction errors belong are first subjected to Huffman coding, and which 
values in each group correspond to the respective prediction errors are then represented by using equi- 
length codes. 

The conversion coefficients other than the above DC components, i.e., the alternating current conver- 

35 sion coefficients (hereinafter referred to as AC components), are supplied to a zigzag scan circuit 106 in 
which the AC components are scanned in a zigzag manner with two-dimensional frequencies from low- 
frequency component to high-frequency component as shown in Fig. 4. followed by Huffman coding in a 
Huffman encoder 107. More specifically, those conversion coefficients for which the quantized outputs are 
not 0 (hereinafter referred to as significant coefficients) are divided into groups depending on their values. 

40 The ID numbers of the divided groups and the numbers of those conversion coefficients which are 
present between every two significant coefficients and for which the quantized outputs are 0 (hereinafter 
referred to as insignificant coefficients) are subjected in pair to Huffman coding. Subsequently, which values 
in each group correspond to the significant coefficients are represented by using equi-length codes. 

Respective code strings of the DC components and the AC components are multiplexed in a 

45 multiplexer 108 and outputted from an output terminal 109. 

Next, when the digital data of the chrominance signals P B . Pr are inputted through the input terminal 
101 the control circuit 302 controls the quantization matrix generation circuit 111 to generate quantization 
matrices for the chrominance signals P 8 , Pr- The switch 114 is changed over to select the side b for 
supplying the quantization matrices for the chrominance signals to the multiplier 110. Subsequently, the 

so chrominance signals processed in a like manner to the luminance signal Y as mentioned above for Huffman 
coding. 

When the compression rate requires to be increased (for a higher degree of compression) from 
limitations in a transmission path or other reasons, control information is entered from a input terminal 303 
so that the control circuit 302 controls a scaling factor generation circuit 301 to increase the scaling factor 
55 S. Conversely, in order to obtain a high-quality image, the scaling factor S is made smaller. 

However because the scaling factor S of the same value is used when coding the luminance signal Y 
and the chrominance signals P 8 . Pr in the above prior art. there has accompanied a shortcoming that image 
quality in color areas (particularly in a red area to which human eyes are more sensitive) deteriorates to a 
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remarkable extent, when the value of the scaling factor S is increased to obtain a high degree of 
compression. 

The present invention is concerned with providing an image processing apparatus which can realize 
satisfactory reproducibility of colors. 

5 Accordingly, in one aspect the present invention comprises input means for inputting image data 
representing a color image, first encoding means for encoding the image data by using a first encoding 
parameter and second encoding means for encoding the same image data by using a second encoding 
parameter different from the first encoding parameter, and wherein said first and second encoding means 
encode the image data in parallel. 

io From a- second aspect the present invention comprises an input step of inputting image data 
representing a color image, a first encoding step of encoding the image data by using a first encoding 
parameter and a second encoding step of encoding the same image data by using a second encoding 
parameter different from the first encoding parameter, wherein said first and second encoding steps are 
performed in parallel. 

is In order that the present invention may be more readily understood, embodiments thereof will now be 
described by way of example and with reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 Fig. 1 is a block diagram showing a first embodiment of the present invention; 

Fig. 2 is a block diagram showing a second embodiment of the present invention; 
Fig. 3 is a block diagram showing the prior art; 
Fig. 4 is a diagram representing a zigzag scan; 

Fig. 5 is a diagram showing one example of quantization matrix elements; and 
25 Fig. 6 is a block diagram showing an application example of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

In the following embodiments of the present invention, a coefficient factor to be multiplied by 

30 quantization matrix elements for a chrominance signal is controlled depending on a scaling factor value So 
of a luminance signal Y, thereby reducing an extent of deterioration in image quality even when the scaling 
factor value is increased for a higher compression rate. 

Fig. 1 is a block diagram showing a first embodiment of the present invention. In the drawing, denoted 
by reference numeral 101 is an input terminal for digital image data, 102 is a DCT ciruit per N x N pixels, 

35 103 is a linear quantizer for linearly quantizing conversion coefficients, 104 is a one-dimensional predictor 
for DC components, 105 is a Huffman encoder, 106 is a zigzag scan circuit for scanning AC components in 
a zigzag pattern, 107 is a Huffman encoder, 108 is a multiplexer for multiplexing codes of the DC and AC 
components, and 109 is an output terminal. 110 is a multiplier for multiplying quantization matrix elements 
for luminance or chrominance by a scaling factor, 111 is a quantization matrix generation circuit for 

40 generating quantization matrices for luminance and chrominance, 112 is a control circuit for controlling the 
quantization matrix generation circuit and a scaling factor generation circuit, 113 is a scaling factor 
generation circuit for generating scaling factors or coefficients to be multiplied for a luminance signal Y and 
chrominance signals P B , Pr. 114, 115 are select switches, and 116 is an input terminal through which 
control information (such as an amount of coded data) is entered to the control circuit. 

45 Note that components having the same functions as those in Fig. 3 are denoted by the same reference 
numerals. The following description will be made about only different parts between the prior art and this 
embodiment. 

The scaling factor generation circuit 113 is controlled by the control circuit 112 to generate the scaling 
factor value So to be multiplied by the quantization matrix elements for the luminance signal Y and, at this 
so time, the switch 115 is changed over to select the contact a. When conversion coefficients of the 
chrominance. T signal P 8 are linearly quantized by the linear quantizer 103 after coding the luminance signal 
Y, the scaling factor generation circuit 113 is controlled by the control circuit 112 to generate a scaling 
factor S B for P 8 , and the switch 1 15 is changed over to select the contact b. In this case, S B is generated as 
a function of So. for example, as expressed below: 

55 

S B = f B (So) 

When conversion coefficients of the chrominance signal Pr are linearly quantized by the linear 
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quantizer 103, the scaling factor generation circuit 113 is controlled by the control circuit 112 to generate a 
scaling factor S R for P R , and the switch 115 is changed over to select the contact c. In this case, S R is 
generated as a function of So, for example, as expressed below: 

5 S R = f R (So) 

Example of function forms of S B , S R usable in this embodiment are enumerated below: 
i) S B = fe(So) = So + XB, 
S R = f R (So) = So - XR 



10 



where XB, XR are positive constants 



20 



S Q x KB 2 (S Q < 0) 



S R = W = ) S 0 X KR 1 (S 0 * 0) 



[S Q X KR 2 (S Q < 0) 



where KBi , KB2, KRi , KR 2 are constants 
25 iii) S B = fs(So) = So x K B (So), 
S R = fft(So) = So x K R (So) 

where K B (So), K R (So) are functions of So 
iv) 

30 

S B - f B<V = S 0 

S R - f R (S 0> = ! S 0 (S 0 < A) 
35 [C (S Q > A) 



where C, A are constants 

Optimum ones of the above functions and constants are selected depending on the compression rate 
40 required and the nature of an input image so that deterioration of image quality is minimized. This selection 
can be achieved by, for example, repeating the coding process several times with different quantization 
characteristics or performing the coding processes with different quantization characteristics in parallel as 
explained later. 

As an alternative method of controlling respective quantization steps for the chrominance signals P B , P R . 
45 the coefficients to be multiplied by the quantization matrix elements for chrominance signals may be 
themseleves controlled rather than generating the scaling factors. In this case, the scaling factor generation 
circuit 113 generates coefficients M B , M R to be multiplied for the chrominance signals P B . Pr in place of the 
scaling factors. M B , M R are functions of So as expressed below: 

50 M B = g B (So), M R = g R (So) 

Example of function forms of M B , M R usable in this case are enumerated below: 
v) 
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M R = W = 2 S ° - TR 



where TB. TR are positive constants 
vi) 



M B = g B (S 0 } = CB x 2S °' 
M R = 9 R (S Q ) =CRx 2 S ° 



where CB, CR are constants 

vii) 



M B = g B (s 0 ) = 2 S 0 x C B (S Q ), 
M R " W " 2S ° X C R ( V' 



25 where C B (So), C R (So) are functions of So 

viii) 



M B = W 



ICO <S 0 > B) 



Optimum ones of the above functions and constants are selected depending on the compression rate 
required and the nature of an input image so that deterioration of image quality is minimized. 

The compression rate can be changed by entering control information to the control circuit 1 1 2 through 
the input terminal 116 and changing the scaling factor value So- The mode for the method of deciding the 

40 quantizing steps for the chrominance signals P g , Pr can also be changed by entering mode information to 
the control circuit 112 through the input terminal 116. 

As mentioned above, since the scaling factor values of the luminance signal Y and the chrominance 
signals Pa, Pr are set by controlling the scaling factor values of the chrominance signals Pb, Pr depending 
on the scaling, factor value So of the luminance signal Y (such that in need of increasing the compression 

45 rate, a smaller scaling factor value is set for Pr which has higher visual sensitivity and a larger scaling factor 
value is set for P B ), an image with less deterioration in image quality can be obtained even when the 
compression rate is increased. Further, by controlling the coefficients to be multiplied by the quantization 
matrix elements for the chrominance signals using function such as f B . f R , gB. gR. the coding can be 
performed more efficiently than the case of setting those coefficients individually. 

so Moreover, even with the image data having other input form (such as GBR inputs) than the foregoing, 
the similar operating effect can also be achieved by weighing the coefficients to be multiplied by the 
quantization matrix elements in a like manner to the above case in match with the visual sensitivity of 
human eyes (e.g., such that in the case of GBR inputs, weighing is applied to the G input near a luminance 
signal, the B input and the R input in this order). 

55 
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(Another Embodiment) 

Fig. 2 shows a second embodiment of the present invention. Components having the same functions as 
those in the prior art are denoted by the same reference numerals and will not be described here. Blocks a. 

5 b and c circumscribed by. broken lines all function in a like manner. However, an input to the block a is only 
the luminance signal Y, an input to the block b is only the chrominance signal P B . and an input to the block 
c is only the chrominance signal P c . Thus, the luminance signal and the chrominance signals are separately 
processed in parallel for coding. Accordingly, a control circuit 222 supplies a control signal suitable for 
parallel processing to a quantization matrix generation circuit 221 so that the quantization matrices for the 

w luminance signal and the chrominance signals are generated almost at the same timing. Likewise, a scaling 
factor generation circuit 220 is controlled by the control circuit 222 to generate scaling factors S Y , S B . S R for 
the luminance signal Y and the chrominance signals P B . Pr almost at the same timing. S Y . S 8 , S R are 
functions of So as expressed below: 

is S Y = So. S B = f B (So), S R = tr(So) 

Alternatively, the scaling factor generation circuit 220 may generate coefficients to be directly multiplied 
by the quantization matrix elements for the chrominance signals. In this case, coefficients M e , M R used for 
the chrominance signals M B , M R are given by: 
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M B = g B (So). M R = g R (S 0 ) 



By so arranging, it is possible to not only obtain an image with less deterioration in image quality even 
at the increased compression rate, but also adapt to high-speed coding through parallel processing, which 
25 permits real-time coding of moving images and the like. 

As explained above, by setting the scaling factor values or the coefficients themselves to be multiplied 
by the quantization matrix elements for the luminance signal Y and the chrominance signals P B , P R 
separately, and controlling the scaling factor values or the coefficients themselves to be multiplied by the 
quantization matrix elements for the chrominance signals P B , P R depending on the scaling factor value So of 
the luminance signal Y, there can be obtained an image with less deterioration in image quality correspond- 
ing to the compression rate (namely, an image with less deterioration in quality of color images can be 
obtained even at the increased compression rate through such a control, for example, that in need of 
increasing the compression rate, a smaller scaling factor value than that of the luminance signal Y is set for 
the chrominance P R which has higher visual sensitivity and a larger scaling factor value is set for the 
chrominance signal P B >. Further, by controlling the coefficients to be multiplied by the quantization matrix 
elements for the chrominance signals using functions, the coding can be performed more efficiently than 
the case of setting those coefficients individually. 

Thus, according to the present invention, it is possible to provide an image encoding apparatus and 
method with which a reproduced image of high quality can be obtained. 

An example of the present invention applied to a moving image encoding apparatus will be described 
below in detail. Fig. 6 is a block diagram of the apparatus. 

In the drawing, denoted by 20 is an input terminal for an analog TV signal. The TV signal sensed by 
using a CCD sensor and inputted through the terminal 20 is converted by an A/D converter 22 into a digital 
signal of 8 bits. The digital signal is divided into blocks each comprising (8 x 8) pixels by an (8 x 8) block 
formation circuit 26 which operates in a like manner to the (N x N) OCT circuit 102 in Fig. 1, the blocks 
being supplied to a DCT circuit 26 one by one. 

Pixel data D„ to Dss of each block are converted by the DCT circuit 28 into a data matrix X„ to Xa 8 
for two-dimensional frequency regions as with the case of Fig. 1, followed by being supplied to quantization 
circuits 32a to 32d and a 1 F (one-frame) delay circuit 34. 

A quantization matrix generation circuit 36 generates similar matrices to those generated from the 
aforesaid quantization matrix generation circuit 111. 

Supplied to multipliers 38a to 38d are initial coefficients C, to C* as control coefficients (scaling factors 
C from a coefficient calculator 48. As with the case of Fig. 1, the multipliers 38a to 38d multiply outputs of 
the quantization matrix generation circuit 36 and the control coefficients C x for respective components, i.e.. 
(W, x Cx/Co), the multiplied results being outputted to the quantization circuits 32a to 32d. The quantization 
circuits 32a to 32d respectively quantize conversion coefficients with the quantizing steps controlled by the 
control coefficients C, to C* and output the quantized values to VLCs (Variable Length Coders) 40a to 40d. 
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In this embodiment, the VLCs 40a to 40d do not output the actual coded data, but output only 
information nb1 to nb4 of total bit number (or each image as resulted from the similar processing to an 
ordinary VLC. 

Using the information nbl to nb4 outputted from the VLCs 40a to 40d and the control coefficients Ci to 
5 C4. the coefficient calculator 48 calculates a control coefficient Cs desired for the total bit number and 
outputs it to a multiplier 38e. 

The coefficient calculator 48 also outputs control coefficients Ce. C 7 as an approximate value of Cs to 
multipliers 38f, 38g and, on the other hand, outputs the control coefficients Cs to C 7 to terminals 44a to 44c, 
respectively. 

10 It is here assumed that when C n £ Cs £ C n+1 holds for Cs, C* and C; meet the relationships C n £ C 6 < 

Cs and Cs < C7 S C n+1 , respectively, and n meets the relationship 1 £ n £ 3. 

The multipliers 38e to 38g respectively multiply outputs of the quantization matrix generation circuit 36 

and the control coefficients Cs to C 7 as with the multipliers 38a to 38d. the multiplied results being 

outputted to quantization circuits 32e to 32g within a vertical blanking period. 
/5 Meanwhile, those DCT conversion coefficients resulted from delaying DCT conversion coefficients 

through one frame by the 1 F delay circuit 34 are inputted to the quantization circuits 32e to 32g. 

The conversion coefficients are quantized in the quantization circuits 32e to 32g with the quantizing 

steps controlled by the control coefficients Cs to C7, and the quantized values are outputted to VLCs 40e to 

40g. In this embodiment, the VLCs 40e to 40g output the actual coded data to entry buffers 42a to 42c and 
20 also output the total bit numbers nb5 to nb7 to a judgment circuit 50. The judgment circuit 50 delivers such 

an output to switches 44 and 46 as to select the quantization result which is less than the desired total bit 

number and nearest thereto. The entry buffers 42a to 42c store the coded data until the judgment circuit 50 

issues the judgment result. The switch 44 selects one of the control coefficients in accordance with the 

judgment result and outputs it to a multiplexer 52. Also, the switch 46 selects one of the coded data and 
25 outputs it to the multiplexer 52. The multiplexer 52 multiplexes the control coefficient and the coded data. 

followed by outputting to an output terminal 54. 

The scaling factor for quantizing the chrominance data may be controlled independently rather than 

being dependent on the scaling factor for quantizing the luminance data as stated before. In other words. 

the luminance data and the chrominance data may be coded in a completely parallel manner. 
30 The scaling factor may be fixed in one frame of an image as mentioned before, or may be varied per 

block of N x N pixels. 

Further, the scaling factor may be controlled depending on the amount of coded data resulted from the 
coding, or may be controlled depending on whether the original image is a line image including characters 
and the like or a natural image. 
35 Additionally, while the component Y is extracted as luminance data and the components P B , Pr are 
extracted as chrominance data in the above embodiments, those components may be (Y, C R , C s ), (L\ a", 
b*), (L\ u*. v*), (Y. u. v) or the like. 

Moreover, the functions of the control circuit 112, the scaling factor generation circuit 113 and the 
quantization matrix generation circuit 111 can be executed by a CPU and a RAM, a ROM, etc. connected to 
40 the CPU. 

Itshould be understood that the present invention is not limited to the embodiments as set forth above 
and may be variously changed and modified within the scope of the invention defined in the attached 
claims. 

45 Claims 

1. A color image processing apparatus comprising: input means for inputting image data representing a 
color image; 

first encoding means for encoding the image data by using a first encoding parameter; and 
50 second encoding means for encoding the same image data by using a second encoding parameter 

different from the first encoding parameter, and 

wherein said first and second encoding means encode the image data in parallel. 

2. Apparatus according to claim 1 comprising: 

55 dividing means for dividing the image data into a plurality of blocks having a predetermined size; 

first quantizing means for quantizing the image data of each block by using a first set of 
quantization parameters; 

second quantizing means for quantizing the same image data of each block by using a second set 
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of quantization parameters different from the first set of quantization parameters, and wherein the first 
encoding means encode the image data quantized by said first quantizing means, and the second 
encoding means encode the image data quantized by said second quantizing means. 

3. Apparatus according to claim 1. wherein the image data is composed of luminance data and 
chrominance data. 

4. Apparatus according to any preceding claim, wherein said input means comprises a CCD sensor. 

5. Apparatus according to any preceding claim, wherein said dividing means divides the image data into 
blocks each comprising N x N pixels. 

6. Apparatus according to any preceding claim, wherein said first set of quantization parameters is smaller 
than said second set of quantization parameters. 

7. Apparatus according to any preceding claim, wherein a second coefficient depends on a first 
coefficient. 

8. Apparatus according to any preceding claim, wherein said apparatus is a moving image coding 
apparatus. 

9. Apparatus according to any preceding claim, wherein said first and second encoding means are 
capable of variable length coding. 

10. A color image coding apparatus according to any preceding claim, further comprising control means for 
controlling a quantization parameter for quantizing the image data by using encoding result of said first 
and second encoding means. 

11. A color image processing method comprising: 

an input step of inputting image data representing a color image; 

a first encoding step of encoding the image data by using a first encoding parameter; and 

a second encoding step of encoding the same image data by using a second encoding parameter 

different from the first encoding parameter, wherein said first and second encoding steps are performed 

in parallel 

12. A method according to claim 1 1 and comprising: 

a dividing step of dividing the input image data into a plurality of blocks having a predetermined 

size; 

a first quantizing step of quantizing the image data of each block by using a first set of quantization 

parameters; , 
a second quantizing step of quantizing the same image data of each block by using a second set 
of quantization parameters different from the first set of quantization parameters, and wherein the first 
encoding step encodes the image data quantized in said first quantizing step and the second encoding 
step encodes the image data quantized in said second quantizing step. 
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